In neural network literature, angular similarity between feature vectors is frequently used for interpreting or re-using learned representations. However, the inner product in neural networks partially disperses information over the scales and angles of the involved input vectors and weight vectors. Therefore, when using only angular similarity on representations trained with the inner product, information loss occurs in downstream methods, which limits their performance. In this paper, we proposed the $\textit{spherization layer}$ to represent all information on angular similarity. The layer 1) maps the pre-activations of input vectors into the specific range of angles, 2) converts the angular coordinates of the vectors to Cartesian coordinates with an additional dimension, and 3) trains decision boundaries from hyperplanes, without bias parameters, passing through the origin. This approach guarantees that representation learning always occurs on the hyperspherical surface without the loss of any information unlike other projection-based methods. Furthermore, this method can be applied to any network by replacing an existing layer. We validate the functional correctness of the proposed method in a toy task, retention ability in well-known image classification tasks, and effectiveness in word analogy test and few-shot learning.

Past efforts to map the Medline database have been limited to small subsets of the available data because of the exponentially increasing memory and processing demands of existing algorithms. We designed a novel algorithm for sparse matrix multiplication that allowed us to apply a self-organizing map to the entire Medline dataset, allowing for a more complete map of existing medical knowledge. The algorithm also increases the feasibility of refining the self-organizing map to account for changes in the dataset over time.

Neural Information Processing Systems http://nips.cc/

For the low-rank RNNs (section 3.1), we trained the

The first "template" vector is During training and evaluation, T is randomly drawn from 1 to 10 in the task. We use three feedforward controller networks coupled with our memory network. For our simulations with BAM, we follow the same controller set-up as above. Unlike in the original work, there is no LSTM controller or reinforcement learning component. There is a reader network and a writer network for the read and write operations respectively.

In our framework, the input is often very noisy, since it corresponds to the converging points of different learning traces. In this section we describe two linear decoders that differ from that in [35] and are more noise-resilient. A.5 is (see [37] for a derivation): p See the T emporal resolutionparagraph below for more details on the discretization of time. A.3 does not impose any explicit constraint on the's in the input vector are A.9 and A.10 is crucial for long temporal horizons, since regularization causes the overall magnitude of the recovered A.3 over the same timesteps as defined by the MP, which provides a direct approximation to the (regularized) Z-transform until a temporal horizon We found this method to be very susceptible to input noise. Figure A.2: The weights of the decoder are trained to minimize the quadratic error between the The decoding method is schematized in Fig. A.2. 's.

Raul Rojas Department of Mathemanullcs and Stanullsnullcs University of Nevada Reno October 2025 Abstract This paper shows how a mulnulllayer perceptron with two hidden layers, which has been designed to classify two classes of data points, can easily be transformed into a deep neural network with one - gate layers and skip connecnullons. As shown in [1], deep one - gate per layer networks can perfectly separate points belonging to two classes in an n - dimensional space. Here, I present an alternanullve proof that may be easier to understand. This proof shows that classical neural networks that separate two classes can be transformed into deep one - gate - per - layer networks with skip connecnullons. A perceptron receives a vector input and divides input space into two subspaces: the posinullve and neganullve half - spaces (Figure 1a).

Hierarchical feed-forward networks have been successfully applied in object recognition. At each level of the hierarchy, features are extracted and encoded, followed by a pooling step. Within this processing pipeline, the common trend is to learn the feature coding templates, often referred as codebook entries, filters, or over-complete basis. Recently, an approach that apparently does not use templates has been shown to obtain very promising results. This is the second-order pooling (O2P) [1, 2, 3, 4, 5]. In this paper, we analyze O2P as a coding-pooling scheme. We find that at testing phase, O2P automatically adapts the feature coding templates to the input features, rather than using templates learned during the training phase. From this finding, we are able to bring common concepts of coding-pooling schemes to O2P, such as feature quantization. This allows for significant accuracy improvements of O2P in standard benchmarks of image classification, namely Caltech101 and VOC07.

We consider the problem of estimating the mean of a set of vectors, which are stored in a distributed system.